miNI : Minimizing Network Interface Memory Requirements with Dynamic Handle Lookup

miNI: Minimizing Network Interface Memory Requirements with Dynamic Handle Lookup Reza Azimi Masters of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto 2002 Recent work in low-latency, high-bandwidth communication systems has resulted in building Network Interface Controllers (NIC) and communication abstractions that support direct access from the NIC to application virtual memory to avoid both data copies and operating system intervention. Such mechanisms require the ability to directly manipulate application buffers in host memory for protection and delivering data. Most modern NICs statically maintain address translation and protection information. However, this results both in high memory requirements for the NIC and limitations in the size of host memory. In this thesis, we categorize the types of data structures for managing communication buffers used in modern NICs, and propose mechanisms to dynamically manage such data structures to alleviate the related limitations. We implement our approach in a modern user–level communication system. The contributions of this thesis are: (i) The integrated approach for dynamic handle lookup that deals with all major lookup data structures reduces NIC memory requirements significantly and eliminates restrictions on the amount of host memory. (ii) A mechanism for handling misses on the receive path by using the retransmission in the NIC. (iii) The detailed performance evaluation with micro-benchmarks and real applications.